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ABSTRACT 


Graphical  Modds  provide  a  pov)«rfiil  tool  f(»  the  formulation  of  general  statistical 
models.  In  a  previous  paper,  n^annoatsos  It  Gdfand,  1991),  the  autnors  argued  that 
samplingi)a8ed  techniques  proviae  a  unifi«i  approach  for  the  analras  of  graphicd  modds 
under  general  distributional  spedfications.  These  techniques  indude  both  noniterative  and 
iterative  Monte  Carlo. 

Our  concern  here  is  with  very  large  graphical  models  whose  siae  and  complexity  may 
prohibit  analysis  within  a  reasonable  time  irame.  Typically  in  large  systems  however, 
interest  focuses  on  the  behavior  of  only  a  few  critical  nodes.  Our  proposal  is  to  devdop,  for 
a  particular  node,  an  approximating  subgraph  which  contains  virtually  as  much 
information  about  the  variable  as  the  full  network,  but  by  virtue  of  its  reduced  size, 
enables  rapid  computational  investigation.  We  provide  an  illustration  using  a  40jiode 
graph.  Thou^  this  is  not  as  large  as  we  would  envision  in  practice,  it  is  convenient  in 
permitting  full  modd  calculations  to  enable  assessment  of  our  approximations. 
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aiding  node).  Cyctei  lead  to  logical  imidaiuildlities  (Peari,  1M6).  Probabilistically,  the 
joiitt  distxibatkm  ci  the  variables  is  not  nniqndy  defined. 

One  of  the  fundamental  idaticms  exi»essible  by  a  DAG  is  that  of  precedence. 
Parental  nodes  precede  their  children,  in  a  causal  or  temporal  sense,  inducing  an  <^ering 
at  the  nodes  in  the  gnqih.  We  enumerate  the  nodes  in  a  top-down  fashion,  starting  with 
those  without  parents  (source  nodes)  at  the  first  level,  proceeding  to  their  children  at  the 
next  level,  and  so  on  to  the  bottom  of  the  graph.  Nodes  allocated  to  the  same  levd  can  be 
ordered  arbitrarily  permitting  many  equivalent  oinmerations.  We  also  define  ihR  parental 
set  of  i  as  pa(i),  and  the  set  of  children  of  i  as  ch(i)  and  denote  the  variable  at  node  i  by  X^. 

Recall  that  two  (possibly  vector-Hralued)  random  variables  and  X2  are 
conditionally  independent  given  a  third  random  variable  X^,  denoted  by  X^^  il  X^lXj.  iff 
f(xj,X2|x3)  *  f(xi|x3)  <(x2l*3)» 

f(x2|x^,X2)  ~  f(x2|xj).  Implicit  in  this  definition  is  that  use  of  conditional  independence 
refines  and  qualifies  general  dependence  rdations.  Conditional  indqiendence  arises 
naturally  for  a  DAG  in  terms  of  the  set  of  predecessors.  That  is,  if  nodes  i  and  j  are  not 
connected  and  i'<j  (i  precedes  j),  then 

jJJllw(H)  <=>  jJJ|P»(j) 

where  pr(j-4)  is  the  set  of  predecessors  of  j  exduding  i.  This  is  equivalent  to  saying  that  j 
is  conditionally  independent  of  i  given  its  parents. 

As  a  result,  the  following  factorisation  of  the  joint  density  of  the  random  variables 
at  the  nodes  in  V  ensues  (Whittaka,  1990,  p.  73).  If  n  is  the  number  of  nodes  in  V, 

f(xi,...,Xjj)  s  n  f(x^|pa(v))  (1) 


2.  Subgr^ih  Approximations 

Modd  reduction  is  hardly  a  novel  statistical  idea  since  parsimony  usually  facilitates 
interpretation.  In  our  case,  however,  the  goal  is  to  produce,  for  a  particular  variable,  an 
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ap]»oximati]ig  snlwet  which  conUiiis  essentially  as  much  infarmation  about  the  variable  as 
the  full  modd,  but  by  virtue  of  its  reduced  siae,  enaUes  rapid  a)mputatioual  investigation. 

Proxiinity  between  two  variables  in  a  DAG  plays  an  important  role.  We  argue  that 
in  such  hierarchical  structures,  for  a  wide  class  of  measures,  the  information  in  Xj  about  X| 
decreases  monotonically  as  j  moves  farther  away  from  i.  For  a  one  dimensional  chain  we 
show  that,  in  studying  Xj,  one  can  truncate  the  chain  a  certain  number  of  steps  away 
from  i  and  obtain  dose  approximations  to  exact  marginal  and  conditional  distributions  for 
X..  In  a  (two-dimensional)  graphical  structure,  of  course,  there  will  be  many  chains, 
emanating  from  a  particular  node  X.  suggesting  the  concqtt  of  a  radius.  The  radius 
around  a  variable  X|  contains  all  those  variaMes  at  a  constant  number  of  edges  away  from 
X|  (the  length  of  the  radius).  For  instance,  a  subset  of  zero  radius  is  the  node  itself,  a 
subset  of  radius  one  is  ch(i)Upa(i)U{i},  and  so  on  until  the  complete  graph  is  reacquired. 
Figure  2  provides  an  illustration  the  concept  of  the  radius  around  a  node.  In  this  20-node 
network,  node  15  is  our  target  (zero  radius).  Next  to  every  other  node,  the  distance  from 
15  has  been  entered. 


[Insert  figure  2  about  here] 

In  the  next  three  subsections,  to  support  the  use  of  subgraph  approximations,  we 
present  theoretical  evidence  asserting  the  d^eneration  of  information  transmitted  from 
nodes  increasingly  distant  from  the  node  of  interest. 


2.1  Infonnatkm  and  Divergence 

We  recall  the  itllback-Leibler  inforMtion  divergence  between  two  measurable 

functions  f(x)  and  g(x)  with  respect  to  an  absolutdy  continuous  measure  u,  defined  as 

f(X) 

g(X) 


t(X)  ,  , 

y  W  =  —  =  S  lot  I 
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lBfDfmalio&  diTergence  liu  the  foUowiog  pfopertiet  (Whittaker,  1990,  p.  94-99). 

(i)  TIm  diTeigeiice  if  well  defined.  That  if,  the  int^ral  in  /^(f;g)  existf,  even 
thoni^  it  might  be  •. 

(H)  The  divergence  if  additive  over  independent  vaxiablef.  Snppoae  that  X,  Y  are 
independent  random  variaUea .  Then 

(iii)  The  information  divergence  if  positive  definite^  that  is  >  0,  with 

equality  holding  if  and  only  if  g(x)a:f(z),  ezcq>t  possibly  on  a  set  of  measure  zero. 

(iv)  The  information  divergence  is  aot  spmsetrie.  That  is,  i^(f:g)  #  A 

symmetric  version  is 

which  we  call  the  aean  information  divergence  between  fonctions  f  and  g  (Kullback  k 
Ldbler,  1951). 

A  particular  information  divergence  is  the  information  proper,  i.e.  the  information 
divergence  testing  omditional  indq>endence  defined  by 

MXjJLXj)  -  y VA) 

The  information  proper  is  symmetric  in  and  X2,  and  achieves  its  minimum  when  X^ 
and  X2  are  indqiendent. 

We  now  investigate  the  behavior  of  the  information  proper  in  hierarchical 

structures.  C!onsider  the  hierarchical  chain  X....-*Xw-».-.-»X.-t...-»X<.  The  information 

n  j  1  1 

contained  in  a  variable  about  others  on  the  chain  is  not  ccmstant.  Intuitivdy,  it  should  be 
larger  for  variables  closer  to  it  than  it  is  for  variables  farther  away.  We  now  prove  that 
this  is  so. 
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TjBBilUt  1.  Ck>iiii<ter  tlie  hieraidiical  leqnace  Xj-»X2-*X^.  For  convex  Innctions  r 


mtr  we  have 


Proof  {DeGtoot  and  C!od,  1986).  Qeariy, 

^(*l)  ^(*2^ 

Therefore, 

K^l\^)  .f(xi|x3)f(x2|x3)  f(xgix^)f(x2|x3) 

f(xi)  ^  f(xj)  fCxj) 

Since  r  is  defined  on  we  have 

by  Jensen’s  inequality.  Switching  the  order  of  integration  between  X2  and  proves  the 
lemma. 


CkaeOaiy  1.  In  a  hierarchical  sequence,  X^-*...-*Xj-»...-»X.-»...X^,  we  have,  for  a 
convex  function  r,  defined  on 


(XllXj 


[X-  X:J 


Proof  Realising  that  fCx^jXi)  s=J'f(xj|x2)..-f(X|_^|xj)dx2...dX|_^  and  that 
do  not  enter  in  the  relation  above,  the  corollaiy  follows  immediately. 

Hence  we  have 


Theosem  1.  In  a  hierarchical  chain  with  Xj^-4Xj^_j-»...-»Xj-»...-»Xj-»...-«Xj 

Inf(Xj JLXj.)  <  InffXj  II  X.) 


is  of  the  form  a  convex  function,  from 


Pro§f. 


ooroilary  1  we  obtain, 


Integrating  both  sides  with  respect  to  f(xp  produces  the  desired  result. 

We  interpret  theorem  1  as  fidlows:  X.  transmits  less  infrmnation  to  than  to  X., 
i.e.  less  to  a  further  node  than  to  a  closer  one.  Whittaker  (1990,  p.  109)  captures  this  in  a 
dightly  different  way. 


Gotollaiy  2.  In  a  hierarchical  chain  we  have, 

MXjJLXj)  *  InfCXjJLXjlXg)  +  InftXjJLXg) 

Proof  From  conditional  independence,  it  is  dear,  that  fj^23  =  ^i|2^|3%’ 
the  definition  of  information  proper,  the  desired  rdatimi  is  equivalent  to, 

MXjJLXjlX,)  +  MXjJLXj)  =  -Ip— L 

n|3*2|3*3' 

=  =  EJo,(^|  =  InffX,  n  X.) 

The  interpretation  is  that  InffX^  il  Xj)  <  InffX^  II  X^).  by  the  quantity 
InffXg  II  X^|X2).  which  is,  in  a  sense,  the  "d^eneration”  of  information  from  X^  to  X^ 
through  its  parent  X2.  The  implication  is  encouragement  for  the  subgraph  approximation. 

2.2  Distanoe  Between  Bistribntiaiis 

Suppose,  as  an  alternative  to  the  information  divergence  between  two  densities,  we 
consider  the  distanoe, 

P-«ll=/lH|d/>  (3.2) 

Consiifor  again  the  Markov  sequence  X^-4X^_^>»,..->X^.  The  daim  is  that,  in 
calculating  marginal  and  conditional  distributions  for  a  particular  X^  it  is  unnecessary  to 
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retreat  back  to  in  order  to  recover  most  of  the  pertinent  information  about  X^. 
Inoeasingiy  distant  predecessors  and  descmidants  should  have  less  influence  than  closer 
ernes.  So  suppose  that  the  chain  is  truncated  a  number  of  steps  away  from  X-.  For  the 
resulting  source  vatiaUes,  we  replace  their  exact  marginal  dmisity  by  an  approximation. 
The  following  theorem  shows  that  in  doing  so,  we  necessarily  better  approximate  exact 
distributions  in  the  sense  as  we  proceed  deeper  into  the  subnetwork. 

Theorem  2.  Suppose  that  X.  and  X^^^  are  neighboring  nodes  in  a  Markov  sequence, 
and  that  all  marginal  and  conditional  distributions  are  defined  in  L^.  Then 

||f(x.H  (Xi)ll  <  l|f(Xj^iH'(Xi^.i)|| 

where 

f(Xi)  =  / 
and 

Proof. 

||1(X,H'(Xi)||  =/|«Xi)-f"(Xi)|<lXi 

=/  /i(*il=S+iW=4+i)'^+i-/ 

^+iK 

by  the  definition  of  the  positive  and  negative  part  functions.  Since  all  terms  are 
nonnegative,  the  absolute  values  can  be  removed,  and,  upon  reversing  the  order  of 
intention. 
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=/K=‘i+i)-''(*i+i)h+i  =  WXi+,)-f'(x,^i)ll 

This  theorem  shows  that  whaterer  density  approximations  are  used  for  the  outer 
edge  of  a  subnetwork,  the  approximations  improve  as  we  move  farther  from  the  edge,  i.e. 
as  we  do  more  processing  incorporating  the  correct  conditional  densities.  We  can  again 
interpret  this  as  supporting  subgraph  approximations. 


2.3.  An  Importance  Sampling  Aignment 

Retaining  the  same  notation  as  in  the  previous  subsections  we  now  argue  that 

f(Xi)  + 

var  ,  '  <var-r  ^ 


(2) 


f(Xi)  MXi+i) 
where,  in  both  cases,  expectations  are  taken  with  respect  to  f  ,  i.e.,  X.  '  f  (X.),  on  the 
lefi^iand  side,  X.^.  *  f  (X|^j|^)  on  the  right-hand  side.  We  interpret  (2)  from  a  Monte 
Carlo  perspective.  On  average,  f  (X.)  is  a  better  importance  sampling  density  for  (better 
matches)  f(X.),  than  f  (X|^^)  is  for  f(Xj^|).  Approximation  improves  as  we  move  further 
from  the  edge  of  the  subnetwork  again  encouragtng  subgraph  approximations. 


Lemma  2.  If  X.  and  are  neighboring  nodes  in  a  Markov  sequence,  then 


nxj) 

f'(Xi) 


=  E 


|Xi 


where  the  expectation  E  is  taken  with  respect  to  the  conditional  distribution  f  (X._^^  |  X^). 

«(Xi|Xi  i)f'(Xi  i) 

Proof.  By  definition,  f  (Xj_j_j  1 3^)  - -v::"". - — — •  But  then, 


E 


f  (Xj) 

I  '*i+P 


f  (Xi^l) 


•'  f'(Xi) 


^i+1  =  - 


f(Xi) 


f  (Xj) 
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neotem  3.  If  X-  and  axe  ndg^boiing  nodes  in  a  Markov  sequence,  then  (2) 

hdds. 

froof.  The  result  is  immediate  frcun  the  Rao-Blackwell  theorem. 

3.  Sinniiati<)[ii  a|ipioadies 

Calculation  of  desired  marginal  and  conditional  distributions  in  a  general  mixed 
directed  graphical  model  requires  high  dimensional  integration/summation.  Such 
calculations  usually  can  not  be  carried  out  analytically,  either  exactly  or  approximately. 
Simulation  methods  offer  a  viable  alternative.  In  particular,  suppose  the  conditional 
information  fixes  a  subset  of  the  variables  (possibly  an  empty  set  if  marginalization  is 
sought),  X^,  of  size  n^  to  specified  levels.  The  nodes  in  X^  are  called  evidence  nodes  in 
Shachter  and  Peot  (1989).  Let  X^  denote  the  a)mplement  of  X^,  i.e.  the  unobserved  nod». 

t 

We  seek  the  conditioiud  distribution  of  X^  given  as  well  as  that  of  X  given 

X^sx^,  where  x'  denotes  a  generic  component  of  X^.  Exact  calculation  of  f(X^ )  X^  s  x^) 
requires  an  n  —  n^  dimensional  integration/summation  and  calculation  of  f(X  |X^  x^) 

requires  an  additional  intention  or  summation  of  dimension  1*  Envisioning  n  and 
possibly  n^  to  be  large,  such  computation  will  not  be  feasible  by  exact  or  approximate 
analytic  methods.  Hence,  we  turn  to  simulation  strategies.  Such  approaches,  implemented 
in  conjunction  with  subgraph  approximations,  enable  rapid  calculation  of  f(x'  |  X^=x^).  In 
subsections  3.1  and  3.2  we  briefly  describe  two  simulation  approaches.  They  are  discussed 
for  a  general  DAG  which  we  envision  as  a  subgraph.  The  Gibbs  sampling  approach  is 
implemmited  for  the  example  in  section  4. 

As  observed  in  Smith  and  Gdfand  (1992),  there  is  an  essential  duality  between  a 
sample  and  the  density  (distribution)  from  which  it  is  generated.  Clearly  the  density 
generates  the  sample.  But  conversely,  given  a  sample  we  can  approximately  recreate  the 
density  and  its  features.  Thus  our  objective  is  to  draw  samples  from  f(X^|X^  =  x^). 
Drawing  observations  from  the  joint  distribution  of  X  is  straightforward.  It  may  be  done 
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in  a  down"  iuliiim  nnng  the  components  of  the  factorisation  (1).  That  is,  we  sample 
all  source  nodes,  then  sample  all  thmr  children,  etc.  The  directed  graphical  model  reveals 
which  samjding  orders  are  thus  feasibk  and  we  shall  in  fact  assunm  that  the  labding  of  the 
X*s  from  to  X^  constitutes  a  feasible  sampling  order. 

In  attempting  to  sample  f(X;^(X^  =  x^)  one  might  take  draws  from  f(X)  and  retain 
those  meeting  the  evidence  X^  =  z^.  Such  rejection  sampling  is  called  logic  sampling  (see, 
e.g.,  Henrion  1988),  and  is  very  inefficient  when  the  nodes  in  X^  are  discrete  but  the  event 
X^=z^  is  rare;  it  is  impossible  if  any  of  the  nodes  in  X^  are  continuous. 

Note  that,  though  we  can  not  obtain  f(X^|X^  =  z^)  explicitly,  we  do  know  its  form 
modulo  a  normalising  constant,  i.e.. 


=  V  •  *o) 

where  the  tight  hand  side  of  (3)  is  given  in  (1).  In  fact  we  can  write 


Xi€X„ 


(3) 


(4) 


Suppose,  as  is  traditional,  that  we  refer  to  what  is  observed  as  "data"  and  what  is 
unobserved  as  "parameters".  Then  the  first  product  on  the  right  side  of  (4)  could  be 
considered  as  a  UkeUhood  since  here  all  the  X|  are  observed  while  the  second  product  could 
be  considered  as  a  prior  since  here  none  of  the  X.  are  observed.  Then  (4)  is  of  the  form 
Likelihood  i  Prior  as  in  customary  Bayesian  modding  and  we  may  bring  to  bear  on  our 
problmn  all  of  the  recently  discussed  sampling-based  technology  for  Bayesian  calculations. 
This  work  includes  noniterative  Monte  Carlo  using  importance  sampling  as  summarized  in 
Geweke  (1989)  as  wdl  as  iterative  or  Markov  chain  Monte  Carlo  as  described  for  Bayesian 
calculations  in  Gdfand  and  Smith  (1990). 


3.1  Indqpendent  or  Noniterative  Monte  Carlo 

Since  we  canimt  sample  from  f(XQ|X^=:z^)  directly,  we  employ  a  suitable 
importance  sampling  density  (ISD).  More  predsdy  fix)m  a  density  say  g(X^)  we  draw  a 
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large  sample  denoted  by  where  g(X^)  has  the  same  support  as 

Expectation  aader  f(X^|X^=x^),  e.g.  E[h(X^)|XQ=xJ,  are  approximated 
by  the  weighted  sum 


m 


(5) 


where  w^  =  f(X^jX^  =  \)/g(y^^)-  Moreover,  resampling  the  X^^  with  probabilities 
g^  3=  w^/Sw^  produces  samples  approximately  distributed  according  to  f(X^  |  X^=x^)  (see 
Rubin,  1988,  Smith  and  Gelfand,  1992). 


The  selection  of  g  becomes  the  primary  concern.  The  more  closely  g  resembles 
f(X^,x^)  the  more  efiSident  g  is  in  terms  of  sample  size  m.  Hence  a  good  ISD  is 
characterized  as  having  fairly  constant  weights  w^.  Such  stability  might  be  measured 
through  the  variance  of  the  w^  (Geweke,  1989)  or  their  entropy  (Ferguson,  1983).  A 


natural  choice  for  g  would  be  the  prior  n  f(X;  I pa{Xj) 

x-ex^  '  'I 


which,  provided  that  it 


is  proper,  it  can  be  readily  sampled  using  a  feasible  sampling  order.  This  choice  of  g  results 
in  weights  w^  =  n  f(X||pa(X|))|  which  would  naturally  be  called  likelihood 

weights  (see  Shachter  and  Peot,  1989),  i.e.,  bigger  weights  are  attached  to  "more  likely" 
Xqi’s.  Such  w^  are  not  likely  to  be  very  stable  since  we  would  not  expect  the  prior  to 
"match"  the  Likelihood  i  Prior  form. 

A  more  refined  selection  of  g  can  be  obtained  as  follows.  Partition  X^  into  a  set  of 

d  c 

disaete  and  a  set  of  continuous  variables  denoted  by  X”  and  X^  respectivdy.  We  consider 
an  ISD  which  samples  equally  likely  over  the  domain  of  X^  and  then,  given  =  x^, 
draws  X^  >  8(^1^  =  ^)'  density  of  X^  under  this  ISD  is  proportional  to  the 

conditional  density  g(X^ I X^).  Thus  to  match  f(XQ,  x^),  for  each  X^  =  x^  we  would 
choose  g(X^|x^)  to  match  f(X^,  x^,  x^).  Methods  for  developing  an  efficient  ISD  for  a 
nonstandardized  continuous  density  have  r^mved  much  attention  lately  (see  e.g.  Geweke, 
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1989,  Oh  &  Berger,  1991,  West,  1991).  In  most  of  this  work  the  resultant  ISD  is  a  mixture 
of  normal  or  t  distributions. 

3,2  Dependent  or  Iterative  Ifonte  dado 

Markov  chain  Monte  Carlo  in  the  form  oi  the  Gibbs  sampler  was  applied  by  Pearl 
(1987),  to  causal  modds  involving  binary  variables,  '^^annoutsos  and  Gelfand  (1991) 
generalize  this  approach  to  arbitrary  directed  graphical  modds.  We  briefly  summarize 
thdr  discussion. 

Implementation  of  the  Gibbs  sampler  to  draw  observations  from  f(X^|X^=x^) 

proceeds  by  making  draws  from  the  univariate  complete  conditional  distributions 

f(X  |X^,x^).  Given  a  starting  state  vector  for  X^,  the  components  of  X^  are  typically 

sampled  in  the  natural  order,  sometimes  referred  to  as  a  raster  scan,  with  the  conditional 

levels  of  X^  updated  after  each  sampling  while  X^  remains  "damped"  at  x^.  One  pass 

through  all  of  the  components  of  X^  is  called  an  iteration.  After  /such  iterations  a  sampled 

vector  X^^  will  result.  Under  mild  conditions  as  l-»  •,  X^^  converges  in  distribution  to  an 

observation  from  f(XQ|XQ=x^).  Such  conditions  mandate  that  we  can  not  permit  any 

purdy  deterministic  nodes  in  our  graphical  modd.  Technically  we  merdy  remove  any  such 

nodes,  adjusting  parents  and  children  accordingly. 

Let  ns  be  more  spedfic  about  the  form  of  the  complete  conditional  distribution  for 
/  /  / 

X  .  It  is  dear  that  f(X  |Xq,X^=x^)  is  proportional  to  (1).  Moreover,  with  regard  to 

/ 

factorization,  only  terms  involving  X  as  a  child  or  as  a  parent  need  be  considered  so  that 

{(x'|x;jc,=x^.(l[x'|pa(x'))  n  fIV|pa(V)])|  (6) 

V€ch(X)  (*n>*o) 

Thus,  the  only  variables  involved  in  the  complete  conditional  distribution  of  X  ,  are  its 
parents,  its  children,  and  the  parents  of  these  children.  This  set  has  been  called  the 
Markov  Uanket  by  Pearl  (1986).  Since  typically  dependence  in  the  modd  is  sparse,  only  a 
few  of  the  terms  in  (1)  appear  in  (6). 
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Turning  to  the  sampling  itself  we  recall  that  by  virtue  of  the  Markovian  updating 
the  conditional  levds  in  (6)  will  change  with  each  draw  of  X  .  In  addition,  the  form  of  (6) 
will  almost  never  be  a  standard  distribution  even  if  the  individual  terms  in  the  product  are. 
For  discrete  variables  sampling  is  routine  upon  standardization/summing  of  (6)  although 
for  ordinal  variables  rejection  methods  are  available  (Devroye,  1986).  For  continuous 
variables  we  might  also  consider  a  rejection  method  (see  Devroye,  1986  or  Ripley  1987). 
For  instance  an  envelope  function  for  (6)  is  Mf{x' |pa(x')]  where 
M=8up  n  f(V|pa(V))|  .  Typically,  f[X  |pa(X  )]  is  readily  sampled  and  M 

is  not  difficult  to  compute  though  it  must  be  recomputed  with  changing  levels  of  X^.  In 
pract''>i  such  envelopes  tend  to  be  very  inefficient  producing  very  small  acceptance  rates. 
This  is  not  surprising  since  the  concentration  of  mass  for  f[x'|pa(X  )]  may  be  quite 
different  from  that  of  (6). 

Alternative  "black  box"  Gibbs  sampler  for  graphical  models  handle  continuous 
variables  by  approximate  inversion  of  the  probability  int^al  transform  as  in,  e.g..  Tanner 
(1991),  by  the  use  of  a  modified  ratio  of  uniforms  method  as  described  in  Wakefield, 
Gelfand  and  Smith  (1991),  by  adaptive  normal  kemd  density  approximations  to  (6)  (see 
Silverman,  1986,  §5.3). 

4.  Ebcample;  Large  NetwoA  Processing  by  Subnetwork  Approximations 

In  this  section,  we  illustrate  subgraph  approximations  to  desired  marginal  and 
conditional  distributions  and  demonstrate  that  with  increasingly  broader  subnetworks, 
approximations  improve.  To  determine  which  variables  are  included  in  a  subnetwork  of 
radius  r  around  a  particular  node,  consider  the  following  simple  procedure: 

(1).  Determine  the  matrix  of  distances  of  every  variable  in  the  network  from  all  others. 
This  is  accomplished  by  determining  the  shortest  path  from  one  variable  to  the 


14 


other  (least  number  of  conditioning  steps). 

(2) .  Choose  a  variable  i  (anyone  will  do  since  this  process  exhausts  the  complete  network 

in  one  pass).  Enter  a  value  of  1  in  all  squares  ij  in  the  grid  where  j€pa(i)Uch(i). 
Enter  a  value  of  2  for  all  nodes  k  in  the  parent  or  children  set  of  each  j  if  a  lower 
value  has  not  been  already  entered  (the  lower  value  is  always  kept).  Repeat  until 
no  nodes  are  left. 

(3) .  Include  in  the  subnetwork  of  the  particular  variable  all  those  variables  that  are 

within  the  given  distance  from  it,  by  consulting  its  line  in  the  above  distance 
matrix. 

(4) .  When  finished,  check  for  cases  where  parental  lists  are  incomplete  and  add  missing 

parents,  accordingly,  to  the  subgraph. 

To  illustrate  why  step  4  is  necessary,  consider  the  case  r=l.  Suppose  a  child  of  the  node  of 
interest  has  a  parent  not  included  in  the  subgraph.  When  we  attempt  to  analyze  the 
subgraph  as  a  DAG,  we  will  be  unable  to  use  this  node’s  conditional  distribution  as  defined 
in  the  full  graph. 

After  step  4  the  reduced  network  will  have  a  set  of  source  nodes.  These  nodes 
require  an  approximate  marginal  distribution  since  the  exact  marginal  is  unknown  and  the 
conditional  distribution  cannot  be  used.  Rough  approximations  can  be  based  upon  a  few 
replications  of  the  complete  network.  In  particular  for  discrete  source  variables  taking  on, 
say,  k  levels,  we  smooth  observed  proportions  by  adding  1/k  to  each  count.  For 
continuous  source  variables  we  use  a  mixture  of  the  conditional  distributions  at  that  node 
based  upon  the  replications  of  the  complete  modd.  Arguments  in  section  2  suggest  that 
with  inoeasing  radius  the  quality  of  these  approximations  becomes  less  consequential. 

In  our  illustrative  4(Miode  network,  nodes  5,  13,  15,  20,  25,  33,  35  and  40  are 
continuous  with  the  remainder  binary.  Linear  logit  models  were  used  for  the  discrete 
variables,  that  is. 
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exp 


P(Xi-l)  = 


(^0+ 

rTTx; 


(?) 


1  +exp(/»0 

in  the  diflciete  case,  with  p(Xj=0)  »  l-p(X|3sl).  In  the  continuous  case  normality  on  the 
l(^-scale  was  assumed  with 


^pa(X.)  ~  ^  ^/i) 

jep»(Ai) 


(8) 


Variances  were  assumed  constant  though,  if  appropriate,  forms  analogous  to  (8)  could  be 
used. 


[Insert  figure  3  about  here] 

A  general  strategy  for  imfdementing  subgraph  approximation  is  as  follows.  Start 
with  a  subnetwork  of  small  radius  and  continue  by  increasing  the  radius  in  a  step  wise 
manner  until  successive  density  estimates  are  stable.  Typically,  satisfactory  estimates  can 
be  obtained  employing  subnetworks  of  radius  hr  smaller  than  the  maximum,  though  this  is 
by  no  means  guaranteed.  When  dependence  is  very  stnmg,  as  information  is  filtered 
through  successive  hierarchical  steps,  it  may  be  that  only  minimal  amounts  are  lost. 

In  figure  3  the  comidete  network  is  jnesented,  with  four  subnetworks  of  node  20 
included  in  figures  4a  through  4d.  The  approximatimis  of  the  marginal  distribution  of  node 
20  based  on  each  subnetwork  are  plotted  in  figure  5. 

[Insert  figures  4ar-4d  about  here] 

They  are  compared  to  the  correct  marginal  distribution  (in  s(did).  We  see  a  gradual 
improvement  of  the  ^proximations  to  the  true  density.  In  Table  1,  the  point  estimates  of 
the  means,  produced  by  the  successive  subnetworks  around  node  20  are  given. 
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[laiert  figon  5  abomt  here] 


Radius 

Estimi^ 

1 

11.6471 

2 

10.6072 

3 

9.6503 

4 

10.4683 

5 

10.3841 

Network 

10.0830 

fthle  1.  Point  estimatOB  of  tite  meam  of  inbgri^  i^ifnozimatioiis  for  node  20. 


Finally,  ibe  sabaetvorlc  ai^zoodmationa  of  tbe  conditional  diatiibntion  around 
node  20  are  plotted.  Nodes  1,  11,  13,  14,  27,  31,  33,  34  were  set  to  fixed  levds.  Each 
snbnetwodc  qtpioximntion  indndes  some  fixed  nodes  while  excluding  others.  As  seen  in 
figure  6  tbe  i^nnoximntlons  based  on  tbe  snlmetworks  of  radius  3  and  4  are  sufficient  to 
obtain  an  adequate  estimate  of  tbe  conditional  distiibnti<m  of  node  20. 

[Insert  figure  6  about  here] 
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ABSTRACT 


Graphical  Modds  provide  a  powerful  tool  for  the  formulation  of  general  statistical 
models.  In  a  previous  paper,  f^Yiannoutsos  ti  Gelfand,  1991),  the  authors  argued  that 
sampling  J)ased  techniques  provide  a  unified  approach  for  the  analysis  of  graphic^  models 
unda  general  distributional  spedfications.  These  techniques  indude  both  noniterative  and 
iterative  Monte  Carlo. 

Our  concern  here  is  with  very  large  graphical  models  whose  size  and  complexity  may 
prohibit  analysis  within  a  reasonable  time  fiame.  Typically  in  large  systems  however, 
interest  focuses  on  the  behavior  of  only  a  few  critical  nodes.  Our  proposal  is  to  devdop,  for 
a  particular  node,  an  approximating  subgraph  which  contains  virtually  as  much 
information  about  the  variable  as  the  full  network,  but  by  virtue  of  its  reduced  size, 
enables  rapid  computational  investigation.  We  provide  an  illustration  using  a  40  mode 
graph.  Though  this  is  not  as  large  as  we  would  envision  in  practice,  it  is  convenient  in 
permitting  fuU  modd  calculations  to  enable  assessment  of  our  approximations. 


